1. Field of the Invention
The present invention relates to MPEG (Moving Picture Experts Group) decoding technology and more particularly to an MPEG decoder, MPEG system decoder and MPEG video decoder which are capable of decreasing the number of circuits required for processing of PTS (Presentation Time Stamp) / DTS (Decoding Time Stamp) values.
2. Description of the Related Art
In an MPEG system, time division multiplexing of an MPEG video stream obtained by encoding image data and an MPEG audio stream obtained by encoding audio data is performed by a sender to form an MPEG system stream for transmission and each of the MPEG video streams and the MPEG audio streams is decoded by a receiver separately to reproduce its original image data and its original audio data. At this point, since the image data and audio data are transmitted in a compressed state, the sender transmits time information about its original data at the same time and then the receiver reconstructs time information of the original data using the above time information and standard time transmitted by the sender to decode and reproduce the image and audio data for outputting.
FIGS. 5A, 5B and 5C are diagrams showing structures of such MPEG data. The MPEG data is hierarchially structured. A top hierarchy is composed of a system layer. A PES (Packetized Elementary Stream) packet obtained by packetizing video data and a PES packet obtained by packetizing audio data being mixed constitutes the system stream as shown in FIG. 5A. Also, as shown in FIG. 5A, the PES packet is composed of a system header and packet data obtained by packetizing video or audio data. The system header is comprised of 4-byte codes including a head starting code having a fixed value (24 bits), a 8-bit start code ID showing a type of data and PTS/DTS values being time information.
The PTS (Presentation Time Stamp) is information used to manage time for outputting the reproduced MPEG image, while the DTS (Decoding Time Stamp) is information used to manage time for starting to decode image data. The MPEG decoder, after adjusting time for reproducing and decoding image by comparing an STC (System Time Clock) value, which is reference time in the MPEG decoder, with the PTS/DTS values, is adapted to perform AV (Audio Visual) synchronization adjustment so as to synchronize image reproducing output to voice reproducing output. The PTS/DTS values are inserted singly or together at an interval of a relatively long time when necessary. Moreover, the PTS/DTS values are inserted both in the case of an MPEG1 system in which a picture is corresponded to a frame of a reproduced image and in the case of an MPEG2-PS system in which the picture is corresponded to the frame or field of the reproduced image.
In the MPEG decoder, each of a video ES (Elementary Stream) and an audio ES is separated from the system layer for processing. The video: ES is formed with video packet data strings in the system stream as shown in FIG. 5B. A plurality of the packet data in the video ES constitutes one picture and picture data contains a plurality of pictures as shown in FIG. 5C. For example, a picture 1 is composed of packet data 1, packet data 2, . . . , Nxe2x88x921, and a picture 2 is composed of packet data N, N+1, . . . . The picture data contains a head starting code composed of a fixed value (24 bits) and 8bit start code ID showing a type of data.
According to specifications of the MPEG system, xe2x80x9c00 to B8xe2x80x9d (in hexadecimal) are used as a start code ID of data in each layer described above to be inserted in the video ES and xe2x80x9cBA to FFxe2x80x9d (in hexadecimal) are used as the start code ID of data in each layer to be inserted in the system stream.
FIG. 6 is a schematic block diagram showing configurations of the conventional MPEG decoder. Operations of the conventional MPEG decoder are hereafter described.
A system decoder 1, when receiving a system stream, decodes a system layer, detects PTS/DTS values and then stores them into a PTS/DTS register 11.
The system decoder 1 is used to output a video stream obtained by decoding the system stream and by separating it. If the video stream is data with the PTS/DTS values added, the system decoder 1 outputs a DTS flag. A memory controller 2 is adapted to store the video stream fed by the system decoder 1 into a memory 3 and, if the existence of a PTS flag is detected, holds an address of data with the PTS added in a PTS/DTS data address register 21.
The memory controller 2, when a video decoder 4 becomes processable, reads the video stream from the memory 3 and outputs it to the video decoder 4 and, if the video stream is data with PTS/DTS values added, outputs the PTS flag to the video decoder 4.
The video decoder 4 is adapted to decode the inputted video stream. At this point, if the existence of the PTS flag is detected, the video decoder 4 informs a CPU (Central Processing Unit) 5 that it has read data with the PTS/DTS values added, by outputting the PTS flag to the CPU 5. The CPU 5, when receiving the PTS flag, checks the address register 21 storing data with the PTS/DTS values added and, at the same time, outputs an instruction to clear the PTS/DTS values held by the PTS/DTS register 11 to the system decoder 1.
The CPU 5 in the MPEG decoder, by comparing an STC value, which is reference time information fed by an STC section 6, with the PTS/DTS values stored in the PTS/DTS register 11, generates an image reproducing output based on decoded video data in accordance with the reference time. For example, if there is a delay or an advance in outputting a reproduced voice and in outputting a reproduced image, the AV (Audio-Visual) synchronization processing is performed by a method employing a deletion (or a skipping process) for every picture or a repetition (i.e., a repeating process) of outputting or by a similar method to synchronize the separately decoded voice reproducing output to the image reproducing output.
In the conventional MPEG decoder as shown in FIG. 6, during a period while the system decoder 1 is performing processing of the PTS/DTS values stored in the PTS/DTS register 11, if newly inputted video stream is data with the PTS/DTS values added, it cannot be processed and must be discarded accordingly. It is impossible to completely process all of the PTS/DTS values, making it difficult to perform the AV synchronization satisfactorily.
In Japanese Laid-open Patent Application No. Hei8-212701, a method is disclosed in which a plurality of PTS values separated from a video stream is sequentially accumulated in a register as a combination with an address and, when the video stream is supplied to a decoding section, the fact is detected, by comparing an address of the video stream with an address read from the registers that the data is one with the PTS value added and by associating the PTS value with a picture, decoding is performed by the skipping or repeating processing.
Moreover, in Japanese Laid-open Patent Application No. Hei10-98699, a method is disclosed in which, by inserting PTS data into an arbitrary detectable position within a data stream such as a video ES or the like and by transferring it to a decoder, fact is detected by a decoder that data is one with the PTS value added.
As described above, the conventional MPEG decoder as shown in FIG. 6 has a problem in that the MPEG decoder, when receiving the input system stream, cannot carry out the processing of the PTS/DTS values if data with the PTS/DTS values added is consecutively contained in the system stream
On the other hand, according to the method disclosed in the above Japanese Laid-open Patent Application No. Hei8-212701, though the processing can be performed with all the PTS data stacked, when the decoding is. performed, the processing must be done after confirming a conformity detected by comparing an address of the PTS value read from the register with an address of the video stream, thus causing the structure of required circuits to be complicated and further an amount of hardware to be increased.
Furthermore, according to the method disclosed in the above Japanese Laid-open Patent Application No. Hei10-98699, since the PTS data is inserted into the data stream and is inputted into the video decoder, though all PTS data can be processed, the vide decoder must perform the processing of detection of the PTS data inserted into the data stream, thus causing the decoding processing to be complicated.
In view of the above, it is an object of the present invention to provide an MPEG decoder which is capable of processing even if an inputted system stream contains consecutive data with PTS/DTS values added and of easily performing PTS/DTS processing and which can be constructed by decreased numbers of circuits required for the PTS/DTS processing.
According to a first aspect of the present invention, there is provided the MPEG decoder including:
a system decoder to separate a video elementary stream from a system stream, to replace a start code ID of picture data with PTS/DTS (Presentation Time Stamp/Decoding Time Stamp) values added in the video elementary stream with a start code ID being unused in a video layer and to output it, said system decoder to hold the PTS/DTS values and the unused start code ID value as an index;
a memory controller to store the video elementary stream fed from the system decoder into a memory; and
a video decoder to issue an instruction of an interruption to a processing device when detecting a fact that the start code ID in the video elementary stream has the unused start code ID at the time of decoding the video elementary stream from the memory controller and to output images obtained by the decoding in accordance with said instruction from the processing device based on the interruption.
In the foregoing, a preferable mode is one wherein the video decoder, when detecting the unused start code ID, issues the instruction of the interruption and outputs the unused start code ID to the processing device and the processing device, in response to the interruption, reads the PTS/DTS values from the system decoder using the unused start code ID as an index and performs processing the PTS/DTS values.
Also, a preferable mode is one wherein the start code ID being unused in the video layer has a value selected out of start code ID values in the system stream.
Also, a preferable mode is one wherein the memory controller, when the video decoder is in a state to be able to perform processing of the video elementary stream, reads the video elementary stream from the memory and outputs it to the video decoder.
Also, a preferable mode is one wherein the processing device, when receiving the instruction of the interruption, by reading the PTS/DTS values having the unused start code ID value as the index from the system decoder and by comparing the PTS/DTS values with an STC value, issues an instruction to the video decoder for outputting images.
Furthermore, a preferable mode is one wherein the instruction to output the images is one to skip or repeat outputting of the images if there is a delay or an advance in reproducing images relative to reference time.
According to a second aspect of the present invention, there is provided an MPEG decoder wherein PTS/DTS values in an inputted system stream are detected and a start code ID is outputted after the start code ID of a picture with the PTS/DTS values added is replaced with a start code ID being unused in a video layer.
In the foregoing, it is preferable that the start code ID being unused in the video layer has a value selected out of start code ID values in a system stream.
According to a third aspect of the present invention, there is provided an MPEG decoder wherein, when a start code ID being unused in a video layer is detected in an inputted video elementary stream, decoding processing is performed by recognizing that it is a start code ID of picture data with PTS/DTS values added.
In the foregoing, it is preferable that the start code ID being unused in the video layer has a value selected out of start code ID values in a system stream.